Skip to content

[Improve](Audit) Auditlog on FE with https enabled#64697

Open
nsivarajan wants to merge 2 commits into
apache:masterfrom
nsivarajan:Auditlog-on-https
Open

[Improve](Audit) Auditlog on FE with https enabled#64697
nsivarajan wants to merge 2 commits into
apache:masterfrom
nsivarajan:Auditlog-on-https

Conversation

@nsivarajan

@nsivarajan nsivarajan commented Jun 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Problem

When enable_https=true and http_port=0 (HTTP disabled), the built-in audit plugin fails silently on every batch. AuditStreamLoader hardcoded http://127.0.0.1:{http_port} at construction time, so the stream load
URL became http://127.0.0.1:0 — an unreachable address. Every loadBatch() call threw a connection error, which was swallowed and counted as discarded logs. No crash, no clear error — audit logging simply stopped working.

Fix

  • Build the stream load URL with the correct scheme (https when enable_https=true) and port (HttpURLUtil.getHttpPort(), which returns https_port when HTTPS is enabled).
  • Apply the cluster CA truststore (mysql_ssl_default_ca_certificate) to the FE connection when the URL is HTTPS, via a new shared utility InternalHttpsUtils.getSslContext().
  • The BE redirect hop (307 Location header) is always plain HTTP and is unaffected — SSL is gated on instanceof HttpsURLConnection.

Also, this serves as preparatory for #60921

Behaviour

Config Before After
enable_https=false http://127.0.0.1:8030 http://127.0.0.1:8030 — unchanged
enable_https=true, http_port=0 http://127.0.0.1:0 — fails silently https://127.0.0.1:8050 — works

Notes

InternalHttpsUtils is introduced here as a shared SSL context utility. A follow-up PR will extend it to cover FE-to-FE internal REST communication over HTTPS.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@nsivarajan

nsivarajan commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

run buildall

@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29243 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6f83514b275f58bd6adc7273703244cbf9f2a72a, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17770	4152	4082	4082
q2	2006	304	192	192
q3	10314	1421	804	804
q4	4687	476	341	341
q5	7506	860	570	570
q6	184	170	137	137
q7	792	840	630	630
q8	9359	1578	1673	1578
q9	6180	4578	4509	4509
q10	6776	1755	1537	1537
q11	455	273	246	246
q12	629	416	287	287
q13	18108	3343	2761	2761
q14	267	270	248	248
q15	q16	791	786	710	710
q17	1036	978	968	968
q18	6804	5727	5585	5585
q19	1301	1342	1085	1085
q20	479	407	259	259
q21	5835	2634	2415	2415
q22	435	372	299	299
Total cold run time: 101714 ms
Total hot run time: 29243 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4474	4321	4357	4321
q2	336	361	225	225
q3	4545	4972	4413	4413
q4	2086	2195	1388	1388
q5	4431	4295	4299	4295
q6	234	173	128	128
q7	1747	1612	2012	1612
q8	2614	2186	2253	2186
q9	8310	8405	7974	7974
q10	4813	4737	4312	4312
q11	558	408	407	407
q12	785	773	551	551
q13	3328	3635	2933	2933
q14	300	291	275	275
q15	q16	710	733	626	626
q17	1388	1370	1356	1356
q18	8015	7216	7256	7216
q19	1157	1169	1150	1150
q20	2232	2218	1945	1945
q21	5328	4628	4488	4488
q22	513	474	394	394
Total cold run time: 57904 ms
Total hot run time: 52195 ms

@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 173951 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6f83514b275f58bd6adc7273703244cbf9f2a72a, data reload: false

query5	4306	636	487	487
query6	446	193	175	175
query7	4811	528	307	307
query8	370	234	199	199
query9	8701	4133	4173	4133
query10	442	328	255	255
query11	5865	2389	2145	2145
query12	153	104	103	103
query13	1274	627	433	433
query14	6402	5495	5487	5487
query14_1	4393	4429	4422	4422
query15	205	199	179	179
query16	989	469	456	456
query17	958	714	580	580
query18	2485	495	360	360
query19	204	188	148	148
query20	112	119	106	106
query21	229	159	121	121
query22	13580	13580	13379	13379
query23	17230	16556	16219	16219
query23_1	16321	16258	16241	16241
query24	7631	1800	1331	1331
query24_1	1344	1337	1327	1327
query25	597	473	400	400
query26	1309	312	179	179
query27	2698	587	357	357
query28	4479	2085	2089	2085
query29	1085	652	486	486
query30	298	242	202	202
query31	1125	1075	954	954
query32	113	64	59	59
query33	552	332	288	288
query34	1214	1281	661	661
query35	745	778	671	671
query36	1377	1382	1243	1243
query37	160	122	92	92
query38	1886	1734	1649	1649
query39	930	920	903	903
query39_1	893	876	878	876
query40	215	123	100	100
query41	63	62	62	62
query42	87	97	86	86
query43	330	331	276	276
query44	1483	768	787	768
query45	202	187	172	172
query46	1126	1205	761	761
query47	2401	2366	2257	2257
query48	401	416	313	313
query49	620	464	373	373
query50	1048	378	263	263
query51	4299	4380	4222	4222
query52	87	81	71	71
query53	255	264	201	201
query54	274	220	211	211
query55	72	70	66	66
query56	227	221	218	218
query57	1427	1414	1306	1306
query58	244	212	195	195
query59	1614	1655	1448	1448
query60	285	242	237	237
query61	158	149	148	148
query62	688	653	592	592
query63	235	188	192	188
query64	2529	780	598	598
query65	4930	4825	4781	4781
query66	1806	460	337	337
query67	29916	29821	29592	29592
query68	3388	1629	968	968
query69	423	306	265	265
query70	1056	979	979	979
query71	299	234	218	218
query72	2906	2680	2313	2313
query73	906	838	458	458
query74	5090	4982	4766	4766
query75	2639	2592	2244	2244
query76	2333	1222	800	800
query77	359	378	280	280
query78	12506	12633	11779	11779
query79	1403	1171	791	791
query80	1267	481	404	404
query81	527	283	244	244
query82	649	162	125	125
query83	324	282	248	248
query84	306	145	112	112
query85	898	501	419	419
query86	430	292	271	271
query87	1851	1847	1803	1803
query88	3721	2820	2772	2772
query89	437	376	327	327
query90	2062	192	183	183
query91	170	169	139	139
query92	66	62	57	57
query93	1653	1418	887	887
query94	715	350	317	317
query95	692	371	344	344
query96	1050	818	353	353
query97	2708	2716	2574	2574
query98	209	205	198	198
query99	1184	1166	1034	1034
Total cold run time: 259586 ms
Total hot run time: 173951 ms

@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 25.28 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6f83514b275f58bd6adc7273703244cbf9f2a72a, data reload: false

query1	0.01	0.01	0.00
query2	0.10	0.06	0.05
query3	0.26	0.14	0.14
query4	1.61	0.14	0.14
query5	0.24	0.22	0.22
query6	1.25	1.05	1.04
query7	0.03	0.01	0.01
query8	0.10	0.04	0.04
query9	0.38	0.32	0.31
query10	0.58	0.54	0.56
query11	0.19	0.14	0.14
query12	0.19	0.14	0.14
query13	0.47	0.48	0.48
query14	1.02	1.01	1.00
query15	0.62	0.59	0.59
query16	0.33	0.32	0.31
query17	1.09	1.13	1.11
query18	0.22	0.21	0.21
query19	2.01	2.01	1.96
query20	0.01	0.01	0.01
query21	15.44	0.23	0.15
query22	4.80	0.05	0.06
query23	16.12	0.32	0.12
query24	2.97	0.41	0.32
query25	0.13	0.05	0.04
query26	0.76	0.20	0.14
query27	0.04	0.05	0.03
query28	3.52	0.91	0.52
query29	12.49	4.33	3.49
query30	0.27	0.16	0.16
query31	2.77	0.58	0.30
query32	3.22	0.58	0.49
query33	3.24	3.23	3.25
query34	15.54	4.19	3.50
query35	3.48	3.52	3.56
query36	0.55	0.45	0.41
query37	0.08	0.06	0.07
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.18	0.16	0.15
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.56 s
Total hot run time: 25.28 s

@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 32.26% (10/31) 🎉
Increment coverage report
Complete coverage report

@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 9.68% (3/31) 🎉
Increment coverage report
Complete coverage report

@nsivarajan nsivarajan marked this pull request as ready for review June 22, 2026 15:53
@nsivarajan

nsivarajan commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

run p0

@nsivarajan

nsivarajan commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

run external

@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 9.68% (3/31) 🎉
Increment coverage report
Complete coverage report

@nsivarajan

nsivarajan commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

run p0

@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 9.68% (3/31) 🎉
Increment coverage report
Complete coverage report

1 similar comment
@hello-stephen

hello-stephen commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 9.68% (3/31) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nsivarajan @hello-stephen